Response to ogtce action of !2: !«9 Jbr appl, 10/780,500 

Remarks 

Claims i~3, 5-11, 13-28 and 32~4i are pending in this applicalion, 
A requ<;st for continued examination under 37 C.F.R. 1 14 is sulimitted 
with this response. 

Claims 22~28> 32-35, 38, and 40-41 stand rejected under 35 U.S.C. 
103(a) as being unpatentabie over Szeiiski et al (US PaKmt Num1:)er 6,157,747, 
herein referred to as Szeiiski) in view of Seago (US Patent Number 5,990,900). 
Claims 1 1, 13-21, 23, and 36-37 stand rejected under 35 U.S.C. 103(a) as 
being ynpatentable over the Gombination of S^ieliski and Seago, and in further 
view of Blank (US Patent Number 5,469,536) . Claims 1-3, 5-6, 8-iO and 39 
stand rejected under 35 U.S.C. i03(a) as being unpatentabie over Szeiiski in 
view of Luken (US Patent Number 5,923,334), and in further view of Seago, 
Claim 7 stands rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Szeiiski, Luken, and Seago, and in further view of Blank. 

Claim Reiectlons- 35 U.S.C ^ lOm) 
Claims 22-28, 32-35, 38, and 40-41 stand rejected under 35 U.S.C. 
103(a.) as being unpatentabie over Sz(;liski et ah (US Patent Number 6,157,747, 
herein referred to as "Szeiiski") in view of Seago (US i-'atent Number 5,990,900, 
herein referred to as "Seago"). 

The rejection of Claims 22-28, 32-35, 38. and 40-41 for obviousness over 
S^cUski m \K\\ ot St\ij3:o must fnll hco isc r^crhc^ Szehski noi 8(\]go t(\]rh<'S 
cTeaUng, a geometric model of an object \^ithm an imajE^e patwiama jucSiiduig 
identifying at le;ast one boundary of the object where the object occupies a field 
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of view In the panorama of greater than 180 de grees, a s required by each of 
Claims 22-28, 32-35, 38, and 40-41. Detaiis fbUaw. 
Claim 22 requires in pertinent part.; 

*(a|.., receiving instructions fi'om a user identitying a. three-dimensional 
geometric surface withiji an Mage panorama, the image paiiorama containing 
en object having one or more textures, the object occupying a field of view of 
more than 180 degrees in the panorama; 

(b) u sing the eomputer creating a geometric mtfjdel of the image 
panoratna based at least in part on the three-dimensional geometric surface 
and the directional vector, wherein creating a geometric model includes 
identifying at least one boundary of the object and using the identified 
boundary to associate geometry' information with the object, the geometiy 
information compiising 3-P coordinates describing the position and orientation 
of the object boundan?- in a reference coordinate system;... (atinotations added). 

likewise, independent Claims 32 and 38 include similar iimitations to 
the cited limitations of Claim 22. 

The subject office action admits that: 

"Szeliski does not expressly suggest: 

•Creating a geometric model includes identifying at least one boundary 
of the object and 

•Using the identified boundary to associate geometry information with 
the otyect, 

oFhe geometry informfttlo'-^ comp/i<i.ng 3-D roordinale^ describing 
the position nnd orientation o' 'he obje^ t f)Ound;ir>' in a reference 
coordmate system." (See, Offict; Action, page 9-10.) 
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Thus, if the combination of Szeliski and Seago teaches iimit.aUon (b) of 
Claim 22, Seago '900 must supply the teaching of modeling the object using an 
identified boundary of the object where the object occupies a field of \=iew of 
more than 180 degrees in the panorama as required by limitation (a) of Claim 
22. However, Seago^s method fails when the object to be modeled occupies 
a Eeld of view greater than 180 degrees in the input image(sj. DetaiLs 
follow. 
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The Iriput to Seago's method Is a '2-D perspective ima^e or a group of 2-D 
images dispki>'ing the object to be modeled. (See, e.g., Seago, col. 1 1, lines 45- 
47). However: 

{1) Because the object requires a field of view of more than 180 

degrees, the object will not fit on a single 2D perspective image. 
Each 2D p(;rsp{;ctive image in the set of images <:an show only a. 
portion of the object, 

(2) Seago's method tor modeiing objects from multiple 2D 
perspective images requires that each 2D perspective image 
show enough matching conjugate features to uniquely determine 
the relative orientation of the 2D perspective images. If the 
images do not contain the required conjugate verti<;es, lines or 
planes, Seago's method fails, (See, Seago *900, coi, 7, lines 29- 
41.) 

(3) Unless the entire object to be modeled fits on each 2D perspective 
image in the set of images, the required matching features a.mong 
the images vcill be absent. 

Thus, Seago *900's method of 3D modeling will fail for any object that 
covers a field of view greater than 180 degrees in the input panorama and the 
combination of Seago *900 with Szeliski '747 will not satisfy the limitations of 
Claim 22 of the subject application. 

Statements 1-3 concerning Seago's method are illustrated in figs. 1-10, 
which are shown below. 
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1- Because the object requires a Held of view of more than 180 degrees, 
the obiect will not fit on a single 2D perspective iiiiage. Each 2D image in 
the set, of imajses will show only a portion of the obiect. 




Fig. 1 

The curved wail is an object that extends beyond 180 degrees in the input 
panoramic image. The lollipop represents the camera position; the wall object 
surrounds the camera position. 
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Fig. 2. 

The camera accjuires 2D perspective images. To create a mosaic similar 
to Szeiisid, the user must take multiple overlapping photos to span the wall's 
extent, which is > 180 degrees. 
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Fig, 4, 
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Fig. 5. 

Without loss of geiierality, the mosaieed im^ge to be use4 comprises three 2D 
images that have sufficient overlap to register the images. 
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Fig. 6. 



But the final mosai<; created with the three initial images cannot be 

displa3?'ed as a single 2D perspective image that presences the features that are 
used to register or model the scene, As shown above, using Szeliski's metliod, 
the image pose cannot change without distorting the original images or 
cropping featiir^^s not visible within pnt 180 degree liiiiit for the 2p perspective 
image required as input for Seago's method. The white portions of the images 
of the wall object illustrate that at least a portion of the wall object will be left 
out of a.ny single 2D perspective image. After all, the wall object spans a field 
of view with more than 1.80 degrees. 
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2. Seago's method for tnoclelmg objC'^ts from multiple 2D perspf^ctivf^ 
images reqnires that each 2D perspectivt; ima^e show enough 
matching conjugate features to uniquely determine the relative 
orientation of the 2D perspective images. If the images do not contain 
the required coniuaatc vertices, lines or planes, Seago's method falls, 
(See, Seago ^900. col. 7. lines 29-41.) 

This statement follows directly from Seago 90Q's disclosure. 
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3. Unless the enti}:e obi'ect to be modeled fits on each 2D perspective 
image in the set M ima^^ required conjug^ate matching features 

among the Images will be absent. 

Taking the simplest example of a plane, consider the process of geometrically 
modeling the wall, which is taken as two overlapping images, A and B. 




Fig. 7. 
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A 



Fig. 8. 

To model a plmie, four points or edges need to be corresponded between 
images A and B. However, because image A contains only a portion of the wail, 
points 3 and 4 that describes the wall's rightward extent must be selected 
arbitrarily and will be unusable for modeling the wall object accurately. 
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Fig. 9. 

Likewise, in imagt; B, verticCvS 3 and 4 can be identified with certainty, but 
vertices .1 and 2 are arbitrary' and, therefore, are unusable for modeling the 
wall with certainty. 
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m 10. 

Fig. 10 illustrates the impossibility of ideritii^nng sufficient, correct 
conj agate points to appiy Seago's method when the entire otyect cannot fit onto 
a single 2D perspective imtige. Foi*ming correspondences between image point 
A~l /image point B-1 and image point A-2/ image point B-2 does not work 
because image point B~l and image point B-2 are arbitrary. Likewise, forming 
correspondences between image point A-3 /image point B-3 and image point A- 
4/image point B-4 does not work because image point A-3 and image point A-4 
ere arbitrary. Because correct conjugate matches cannot foe determined when 
the entire object does not fit on each 2D perspective input image, then the 
simplest case of a wail cannot be modeled using the method disclosed in Seago 
'900. 

Szeliski *774's method can be used to produce a single 2i:) image that has 
four vertices visible to model in the case of the wall shown in figs. 7-10, since 
the wall's extent is not great than ISO degrees. But for objects that require 
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more than a 180 degree field of view, the necessan^ vertices cannot be projected 
onto a single 2D perspective iiiiage to satisfy Seago 'QOO's method. (See iigs. i- 
6 and accompanying discussion above.) If a mosaic/panorama with an object 
extends beyond 180 degrees, the image needs to be warped/remapped to fit 
into a single 2D image. Warping or remapping features does not work for 
Seago's method because Seago 's method relies on pcTsp(^ctiv(^. 

Because step *b" of Claims 22, 32 and 38, ibr objects that occupy a field 
of view greater than 180 degrees as required by step "a^,** is neither taught nor 
suggested by any GGmbina.tion of the teachings of S2s?liski and Seago, a prima 
facie case of obviousness is lacking. Claims 22, 32 and 38 are therefore 
deemed patentable. Claims 23-28, 33-35 and 40-4 i depend from Claims 22, 
32, and 38, respectively and add further limita.tions. Each of thcvse claims is 
deemed nonobvious over Szeliski in view of Seago for at least the same reasons 
as for Claims 22, 32 and 38. 

Claims 11. . 13-2 L 23. and 36-37 stand rejected under 35 U.S.C. 1 03M 
asbelng..unp.atentable..o^^ 

further view of Blank (US Patent Number 5.469,536). 

The rejections of Claims 1 1, 13-21, 23, and 36-37 for obviousness rely on 

Szeliski and Seago for teaching the iimitations cited above for Claim 22 and 32, 
namely; limitation (b) of Claim 22, where the object occupies a field of view of 
more than 180 degrc^es in the panorama as required by limitation (a| of Claim 
22. Therefore, a. prima, facie case of obviousness is lacking for each of these 
eiaims because, as shom^n above, Szeliski and Seago do not teach, disclose or 
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suggest these limitations of the claims. Claims 11, 13-21, 23, and 36-37 ai'e 
therefore deemed patentable over the cited references. 

Claims 1-3. 5-6. 8-10 and 39 stand rejected under 35 U.S.C. I03{a] as 
being unpatentable over Szeliski in view of Luken fUS Patent Number 
5,923,334], eaid in further view of Seago. 

The rejections of Claims 1-3, 5"6, 8- 10 and 39 for obviousness rely on 
Szeliski and Seago for teaching the limitations cited above in rejecting Claim 22 
and 32, namely: limitation {bj of Claim 22, where the object occupies a field of 
view of more than 180 degrees in the panorama as required by limitation (a) of 
Claim 22, Therefore, a prima facie case of obviousness is lacking for each of 
these ciainis because, as shown above, Szeliski and Seago do not teach, 
disclose or suggest these limitations of the claims. Claims 1-3, 5-6, 8- 10 and 
39 are therefore deemed patentable over the cited references. 

Claim 7 stands rejected under 35 U.S.C. i03(a) as being unpatentable 
over the combination of SzeHsld. Luken. and Seago. and in further view of 
Blank. 

The rejection of Claim 7 for obviousness relies on Szeliski and Seago for 
teaching the limitations cited above in rejecting Claim 22 and 32, namely: 
limitation (b) of Claim 22, where the object occupit^s a field of view of more than 
180 degrees in the panorama as required by limitation (a) of Claim 22. 
Therefor^;, a prima facie cast; of obviousness is lacking for Claim 7 because, as 
shown above, Sz;eliski and Seago do not teach, disclose or suggest these 
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littiitations of the claims. Glaim 7 is therefore deemed patentable over the cited 

references. 

Applicant requests reconsideration of all pending claims and a notice of 
allowance. The Examiner is requested to telephone the undersigned if any 

matters remain outstanding so that they may be resolved expeditiously . The 
Cominissioner is hereby authorized to charge any deficiency in the fees lii«;d, 
asserted to be filed or which should have been filed herewith to our Deposit 
Account No, 19-4972. 

Respectfully submitted, 
/John L- Conway, #48,241/ 
John L, Comvay 

Sunstein, Kann, Murphy & Timbers LLP Registration No. 48,241 
125 Summer Street Attorney for Applicant 

Boston , MA 02 1 10- 16 18 
(617) 443-9292 

<y?MH /OO 108 1 2630 1 J . Sdaw 
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